Papers with argumentative reasoning
Evaluating the Performance of Large Language Models via Debates (2025.findings-naacl)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) are evolving and impacting various fields . current methods for evaluation are based on fixed, domain-specific questions or rely on human input, making them unscalable. |
| Approach: | They propose a benchmarking framework based on debates between LLMs, judged by another LLM. |
| Outcome: | The proposed framework achieves rankings that align closely with popular rankings based on human input eliminating the need for costly crowdsourcing. |
Natural Language Reasoning in Large Language Models: Analysis and Evaluation (2025.findings-acl)
Copied to clipboard
| Challenge: | Argumentative reasoning presents unique challenges due to its reliance on context, implicit assumptions, and value judgments. |
| Approach: | They propose a large-scale evaluation of LLMs' unconstrained natural language reasoning capabilities . they formalise a new strategy designed to evaluate argumentative reasoning in LLM . |
| Outcome: | The proposed model performs better on a range of reasoning tasks than other models. |
Beyond Recognising Entailment: Formalising Natural Language Inference from an Argumentative Perspective (2024.acl-long)
Copied to clipboard
| Challenge: | Existing methods for recognizing textual entailment lack a standardized definition of inference, making it difficult to compare methods trained on different datasets. |
| Approach: | They propose a rigorous approach to align entailment recognition with argumentation theory by using a tool to assist humans in annotating arguments according to the PTA. |
| Outcome: | The proposed model is based on a human-trained dataset and provides insights into non-expert annotator training. |